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Method for automatic segmentation of medical images 



(57) An method for automatically extracting a three- 
dimensional sub-structure, for example a coronary ves- 
sel tree, from a plurality of slice images comprises the 
steps of depositing a seed point within a selected region 
of interest and segmenting the plurality of slice images 
responsive to the deposition of the seed point. The seg- 
mentation is performed in accordance with a plurality of 



predetermined classification values to extract the three- 
dimensional sub-structure. A method for automatically 
segmenting comprises the steps of receiving a depos- 
ited seed point, extracting a plurality of desired regions 
within the sub-structure and generating a segmented 
sub-structure from the extracted desired regions. 
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Description 

[0001] The present invention relates to image 
processing and more specifically to segmenting three- 
dimensional (3D) structures from a series of cardiac im- 
ages. 

[0002] Computed tomography (CT) scans provide a 
non-invasive method for imaging the human body. One 
particular area of interest is imaging the heart and the 
vessels contained in the heart. Doctors are especially 
interested in examining the coronary arteries because 
they provide the doctor with a measure of the patient's 
cardiac health. 

[0003] Medical images of a subject's heart typically 
are comprised of several slices throughout the 3D vol- 
ume of the heart. In addition, the heart may be imaged 
at several different instants or phases in the cardiac cy- 
cle. Thus the ensemble of all images gives a picture of 
the heart during the complete course of one heart beat. 
[0004] It is often desirable to use the set of images to 
extract quantitative information about the heart motion 
useful in medical diagnosis. Such information includes 
measurement of the cardiac blood volume, the ejection 
fraction and the amount of wall motion. It is necessary 
to detect and differentiate contiguous anatomical re- 
gions of the heart to perform this measurement. Identi- 
fication of contiguous regions of the same material is 
known as segmentation. 

[0005] Doctors can use existing techniques for exam- 
ining the data provided by a cardiac CT scan, including 
standard two and three-dimensional viewing tech- 
niques. However, if they wish to examine the three-di- 
mensional vessel tree of the heart separately from the 
heart muscle and surrounding tissues, few good options 
exist. Tools exist that allow doctors to extract parts of 
the vessels from the surrounding tissues, but they either 
do not extract the whole vessel tree or they require sig- 
nificant user interaction. Often doctors must tediously 
segment the vessel tree by hand from either axial CT 
images or a 3D rendering of the image data. This may 
take several hours and requires the physician to manu- 
ally define what is part of the vessel tree. What is needed 
is an automated technique for segmenting and display- 
ing justthe coronary vessel tree, which requires reduced 
operator interaction. 

[0006] In accordance with the invention, a method for 
automatically extracting a three-dimensional sub-struc- 
ture, for example a coronary vessel tree, from a plurality 
of slice images comprises the steps of depositing a seed 
point within a selected region of interest and segmenting 
the plurality of slice images responsive to the deposition 
of the seed point. The segmentation is performed in ac- 
cordance with a plurality of predetermined classification 
values to extract the three-dimensional sub-structure. 
[0007] The invention will now be described in greater 
detail, by way of example, with reference to the draw- 
ings, in which:- 



Figure 1 is a block diagram showing an automatic 
extraction method to which preferred embodiments 
of the present invention are applicable; 

5 Figure 2 is a flow diagram showing an automatic 

segmentation method useful in the method shown 
in Figure 1 ; 

Figure 3 is an illustration of an interim image result- 
10 ing from steps in the automatic method of Figure 2; 

Figure 4 an illustration of an image resulting from 
steps in the automatic method of Figure 2; 

15 Figure 5 is an illustration of volumetric input data to 
which preferred embodiments of the present inven- 
tion are applicable; and, 

Figure 6 is an illustration of a resultant image of a 
20 coronary vessel tree after employing the methods 
shown in Figures 1 and 2. 

[0008] Referring to Figure 1, there is shown a block 
diagram of an embodiment of a method for automatically 

25 extracting a three-dimensional coronary sub-structure 
from a plurality of cardiac slice images. The user inter- 
action proceeds as shown in Figure 1 . At 100, an axial 
set of images is acquired, for example by computed to- 
mography (CT) scanning, in a well-known manner. In 

30 the description that follows, a set of axial CT images is 
used as the image data. However, it is to be appreciated 
that image data from Magnetic Resonance Imaging 
(MRI) is also suitable as image data for the automatic 
extraction method of the present invention. Typically, the 

35 desired coronary sub-structure is the coronary vessel 
tree that comprises the ascending aorta and the left and 
right coronary arteries. However, it is to be appreciated 
that the extraction method is suitable for other sub-struc- 
tures within the heart, or alternatively within other dy- 

40 namic organs within the body. 

[0009] The user (for example, a physician or techni- 
cian) is presented with axial CT images from a cardiac 
scan at 1 01 . At 1 02, the user examines the image data 
to locate a region of interest (ROI). In this embodiment, 

45 the region of interest is the ascending aorta. The user 
moves through the images slice by slice, until the as- 
cending aorta is seen. At 1 03, the user selects a region 
of interest (ROI), for example the ascending aorta, by 
depositing a seed point. Deposition of a seed point is 

50 performed using a user interaction device, such as a 
computer mouse. Once the seed point is deposited, the 
user initiates the execution of the automatic segmenta- 
tion process, which will be described with reference to 
Figure 2. Automatic, as used herewith, refers to a ma- 

55 chine or computer-produced sequence of events versus 
requiring operator interaction. The seed point serves as 
the starting point for the data segmentation. The vessel 
segmentation process is responsive to the single depos- 
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ited seed point, as set forth below in greater detail. 
[0010] The vessel segmentation process automati- 
cally evaluates the image data in accordance with pre- 
determined classification values to extract desired por- 
tions of the data to keep for the vessel visualization and 
to discard portions not desired. These classification val- 
ues include pixel intensity or CT number of the pixels, 
pixel location relative to the seed point, the size of sim- 
ilar pixel groupings, and pixel connectivity in both two 
and three dimensions. 

[001 1 ] Referring further to Figure 1 , after execution of 
the automatic segmentation process, an image of the 
extracted region of interest is generated at 105. The 
modified data is displayed using any of the standard vis- 
ualization techniques well known in the art, such as vol- 
ume rendering or surface rendering. Desirably volume 
rendering is used to present a 3D volumetric image to 
the user, however, one skilled in the art would recognize 
that many other visualization techniques are possible. 
[0012] In an alternative embodiment, if multiple vol- 
umes of image data are available for different phases of 
the heart cycle, the segmentation process is desirably 
applied multiple times to produce iterative segmenta- 
tions of the vessel tree at different phases of the heart, 
as shown in 1 06 of Figure 1 . Thereafter, results of these 
iterative segmentations are suitable for animated dis- 
play over time. In a further alternative embodiment, the 
automatically extracted coronary sub-structure is dis- 
played and cross-referenced with axial and reformatted 
images. 

[0013] Referring to Figure 2, there is shown a more 
detailed flow diagram of an embodiment of an automatic 
segmentation method useful in the automatic extraction 
method of Figure 1 . In an embodiment, the vessel seg- 
mentation process comprises a two-part extraction 
process. The first part, shown as 2010 in Figure 2, ex- 
tracts the ascending aorta from the data. The ascending 
aorta is a large vessel that carries blood from the left 
ventricle to the body. The left and right coronary arteries 
branch from the ascending aorta. It is of particular inter- 
est to doctors to see the attachment points of the main 
coronary arteries to the ascending aorta because this 
area can not be easily examined with current imaging 
modalities. The ascending aorta will be kept as part of 
the final output. 

[0014] The second part of the vessel segmentation 
process, shown as 2020 in Figure 2, extracts the left and 
right coronary arteries from the data volume. 
[0015] The results of the two main parts of the proc- 
ess, the ascending aorta and the main coronary arteries, 
are presented to the user in the final output. The two- 
part process (201 0 and 2020) facilitates an efficient iso- 
lation of the ascending aorta and then the coronary ar- 
teries based on respective predetermined classification 
values, thereby removing undesired portions of the 
heart that are not needed for diagnosis. The predeter- 
mined classification values, such as pixel intensity, pixel 
size and connectivity, which will discussed more fully 



with reference to Figure 2. 

[0016] Referring further to Figure 2, the automatic 
segmentation process is responsive to an input volume 
of data at 200. As with the process of Figure 1 , the input 

5 volume is an axial set of images from CT or MRI imaging 
operations. At 201 , the input volume data is evaluated 
against a threshold in a known manner to produce a bi- 
nary mask of pixels that are in the value range of inter- 
est, and those that are outside that range. In this em- 

10 bodiment, the classification value or threshold is a 
measure of pixel intensity often referred to as CT 
number and measured in Hounsfield units. Through ex- 
perimentation and other known manners, a threshold is 
defined to indicate pixel intensity typical for an ascend- 

15 jng aorta. As a result of thresholding at 201 , pixels are 
turned "on" if they are part of a certain range of CT num- 
bers and are turned "off 1 if they are not a part of the range 
of CT numbers. The binary mask resulting from applying 
a threshold to the input CT values is the input to the two 

20 parts of the segmentation process described earlier. 
Figure 3 shows a representative interim image after 
thresholding, showing multiple coronary sub-structures 
which are turned "on" after thresholding. As is shown, 
the multiple coronary sub-structures include more than 

25 the coronary vessels (which cannot be clearly seen) and 
the ascending aorta 302. Also shown is seed point 300. 
Further extraction, which will be set forth below in great- 
er detail, will result in extraction of only the desired sub- 
structures. 

30 [0017] The vessel segmentation process now ex- 
tracts the ascending aorta. After the threshold is applied 
to the original CT data, a seed connectivity process at 
202 in Figure 2, is run using the seed point indicated by 
the user. The seed connectivity searches the binary pix- 

35 el mask output from the thresholding step. All the "on" 
pixels that are connected to the seed point specified by 
the user are kept and pixels that are either "off" or are 
not connected to the seed point are discarded. Connec- 
tivity in this case is defined as any pixels in the same 

40 axial image, which touch each other, either at a corner 
or along an edge. This is known in the art as eight-con- 
nectedness. The output of this step contains the as- 
cending aorta plus some additional areas that need to 
be discarded. These areas will be refined in the addi- 

45 tional processing steps that follow. 

[0018] After running the two-dimensional seed con- 
nectivity, a 7x7x7, 3D erosion, labeled 203 in Figure 2, 
is applied to the output of the connectivity. The 7x7x7 
erosion is a technique, well known in the art, which re- 

50 moves outer layers of a structure. In this case, the 7x7x7 
erosion breaks bridges that connectthe ascending aorta 
to other structures that are not of interest. The output of 
this step is another binary mask. 
[0019] The eight connected seed connectivity 

55 processing, labeled 204 in Figure 2, is now applied to 
the output of the 7x7x7 erosion. This technique exe- 
cutes in the same manner as before, however, the out- 
put results are now more refined because connected 
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structures that are not of interest have been eliminated. 
[0020] The next step in the extraction of the ascending 
aorta is the removal of large islands from the results of 
the priorstep, labeled 205 and 206 in Figure 2. An island 
removal technique is applied that labels "islands" or 
groups of pixels in the axial images that are smallerthan 
a certain size. Size is used as a classification value to 
include or exclude structures. In this embodiment, de- 
sirably the size is set at 7000 pixels. Once the islands 
that are smallerthan 7000 pixels are labeled, all the oth- 
er pixels are removed. This has the effect of keeping 
every grouping of pixels in an axial image that is smaller 
than 7000 pixels. Areas larger than 7000 pixels, for in- 
stance the chambers of the heart, are removed from the 
binary pixel mask. 

[0021 ] After removing the large islands from the bina- 
ry mask, the remaining pixels are expanded using 7x7x7 
dilation, a technique well known in the art, labeled 207 
in Figure 2. The 7x7x7 dilation adds layers to a structure. 
This has the effect of restoring the size of structures that 
are left in the binary mask to their original size. The out- 
put of this step is a binary maskthat defines the ascend- 
ing aorta. "On" pixels are part of the ascending aorta 
and "off" pixels are not. 

[0022] The output of the dilation is subtracted from the 
output of the original thresholding. This is step 208 in 
Figure 2. The pixels that are "on" in both the outputs of 
the previous dilation and the original thresholding are 
turned "off". The other pixels are unchanged. The 
processing thus far has effectively isolated and extract- 
ed the ascending aorta for further processing. 
[0023] The extraction of the main coronary arteries 
begins at 209. The classification values used to extract 
the main coronary arteries are mainly size and connec- 
tivity. Another eight connected, two-dimensional island 
removal is performed on the output of the prior step (208 
of Figure 2). The size threshold for this step is different 
from the previous island removal (205) and is selected 
to mark objects that are too big to be coronary vessels. 
In this step, this size threshold is set to 2000 pixels. The 
output of this step is similar to the output from the prior 
step; however, every cluster of pixels that is less than 
2000 pixels in size has been removed. 
[0024] Afterthe island removal step, a two dimension- 
al dilation is performed with a 5x5 kernel on the output 
of the island removal as in step 210 of Figure 2. This 
step effectively overestimates the size of these objects 
which are bigger than 2000 pixels, by the selection of 
the kernel. 

[0025] The results of the dilation are then subtracted 
at step 211 from the output of the original thresholding 
at step 201 . This step removes the objects from the orig- 
inal thresholding that are too large to be vessels. 
[0026] A dilation at step 212 is applied to the results 
of the subtraction step 211 . The dilation strengthens the 
connectivity of pixels along the axis of the body The axis 
along the body correspond to the direction of the vessels 
within the body. Large portions of the coronary arteries 



run in this direction. In this embodiment, a non-symmet- 
ric, 3x3x5 dilation kernel is used. 
[0027] Next a three-dimensional connectivity proc- 
ess, step 21 3 of Figure 2, is applied to the results of the 

5 previous step. The same seed point specified by the us- 
er is now used to extract all the pixels that are connected 
to the seed. Connected in this case is defined as pixels 
that share an edge, a face, or a corner. As is well known 
in the art, this is called 26-connected. The output of this 

10 step defines the pixels that will be kept as part of the 
desired coronary sub-structure, the coronary vessel 
tree. 

[0028] Finally, a masking operation, step 214 of Fig- 
ure 2, is performed on the original data. The output of 

15 the three dimensional connectivity is one input to this 
step and the original data is the other input. The masking 
at 214 effectively converts the extracted pixels from 
201 0 and 2020 to the respective original CT values. If a 
pixel is "off 1 in the output of the previous step, it is set to 

20 zero in the original data. Otherwise, the pixels in the orig- 
inal data are left alone. Figure 4 shows a representative 
resultant image of masking 214, showing the extracted 
ascending aorta 302 and coronary vessels 301 . 
[0029] Figures 5 and 6 illustrate the effects of the em- 

25 bodiments of the present invention. Referring to Figure 
5, there is shown a volumetric image of a heart including 
an ascending aorta 302 and coronary vessels 301 . After 
using the methods discussed with reference to Figures 
1 and 2, Figure 6 illustrates a resultant image of an ex- 

30 tracted coronary vessel tree that is obtained with re- 
duced operator interaction which can be used for med- 
ical diagnosis. 



1. An method for automatically extracting a three-di- 
mensional sub-structure from a plurality of slice im- 
ages, said method comprising: 

40 

depositing a seed point within a selected region 
of interest on at least one of said slice images; 
and, 

segmenting said plurality of slice images re- 
45 sponsive to said deposited seed point and in 

accordance with a plurality of predetermined 
classification values to extract said three-di- 
mensional sub-structure. 

50 2. The method of claim 1 wherein said sub-structure 
comprises a coronary vessel tree. 

3. The method of claim 1 or 2 wherein said plurality of 
slice images are acquired from computed tomogra- 
ms phy (CT) imaging. 

4. The method of claim 1 or 2 wherein said plurality of 
slice images are acquired from magnetic resonance 
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imaging (MRI). 

5. The method of any preceding claim further compris- 
ing displaying said automatically extracted sub- 
structure using three-dimensional volume render- 5 
ing techniques. 

6. The method of any one of claims 1 to 5 further com- 
prising displaying said automatically extracted sub- 
structure using that is cross-referenced with axial 10 
and reformatted images. 

7. The method of any preceding claim furthercompris- 
ing repeating said segmentation for a plurality of im- 
age data sets which contain respective pluralities of 15 
images from multiple phases of the heart and fur- 
ther displaying respective extracted coronary sub- 
structures as animated overtime. 

8. A method for automatically segmenting a plurality 20 
of image slices for a sub-structure contained with 
said plurality of image slices, said segmentation 
method comprising: 

receiving a deposited seed point; 25 
extracting a plurality of desired regions within 
said sub-structure in accordance with respec- 
tive predetermined classifications values for 
said respective desired regions and connected 
to said deposited seed point; and, 30 
generating a segmented sub-structure from 
said plurality of extracted desired regions. 

9. The method of claim 8 wherein said sub-structure 

is a coronary vessel tree and, said desired regions 35 
an ascending aorta and left and right coronary ar- 
teries. 

10. The method of claim 8 or 9 wherein said predeter- 
mined classification values comprise pixel intensity, 40 
pixel size and connectivity. 

1 1 . The method of claim 8, 9 or 1 0 wherein said plurality 
of slice images are acquired from computed tomog- 
raphy (CT) imaging. 45 

12. The method of claim 8,9 or 1 0 wherein said plurality 
of slice images are acquired from magnetic reso- 
nance imaging (MRI). 

50 
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